Papers with large-scale evaluations
FreeEval: A Modular Framework for Trustworthy and Efficient Evaluation of Large Language Models (2024.emnlp-demo)
Copied to clipboard
Zhuohao Yu, Chang Gao, Wenjin Yao, Yidong Wang, Zhengran Zeng, Wei Ye, Jindong Wang, Yue Zhang, Shikun Zhang
| Challenge: | Large language models (LLMs) have revolutionized natural language processing with impressive performance across various tasks. |
| Approach: | They propose a framework for automated evaluations of large language models . they open-source their code at https://github.com/WisdomShell/FreeEval . |
| Outcome: | The framework is open-source and can be used to develop and validate new evaluation methods. |
Cardiverse: Harnessing LLMs for Novel Card Game Prototyping (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have enhanced the possibilities for game prototyping, but they face significant challenges. |
| Approach: | They propose a graph-based indexing method for generating novel game variations and an LLM-driven system for consistent game code generation validated by gameplay records. |
| Outcome: | The proposed framework accelerates card game prototyping, reduces human labor, and lowers barriers to entry for game developers. |